Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for cudf.Series #4891

Merged
merged 2 commits into from
Sep 26, 2019
Merged

Add support for cudf.Series #4891

merged 2 commits into from
Sep 26, 2019

Conversation

VibhuJawa
Copy link
Contributor

This PR allows cudf.Series to go via columnar initalizlizer .

This pr fixes the code-path to follow when the label is a cudf.Series .

Example to show impact:
X_train = cudf.DataFrame({'1':
                          cudf.Series( np.random.randint(low = -1000, high = 50_000 , size= 1_000_000, dtype=np.int32))})
Y_train = cudf.Series(np.random.randint(low = 0,high = 2 ,size = 1_000_000, dtype=np.int32))

Before this pr below took 2min 18s

%time xgb.DMatrix(X_train,Y_train)

After this PR below takes 10.5 ms

%time xgb.DMatrix(X_train,Y_train)

@trivialfis
Copy link
Member

Thanks for the fix. Will add a test for that.

@trivialfis
Copy link
Member

@VibhuJawa Could you please enable "Allow edits from maintainers." ? Or apply the following patch:

0001-Test-from-series.zip

@trivialfis @VibhuJawa
Test from series.
d0c555f 
Signed-off-by: vibhujawa <vibhujawa@gmail.com>
@VibhuJawa
Copy link
Contributor Author

@VibhuJawa Could you please enable "Allow edits from maintainers." ? Or apply the following patch:

0001-Test-from-series.zip

I think allowing edit from maintainers should be on. Anyways added the patch.

Thanks a lot for your help.

image

@trivialfis
Copy link
Member

@RAMitchell I think I really need to bring back the profiling branch for both CPU and GPU. Our memory usage issue needs to be solved, or at least be monitored.

@RAMitchell
Copy link
Member

@trivialfis can you get what you need from

xgboost/src/common/device_helpers.cuh

Line 212 in 82ee231

class MemoryLogger {

@trivialfis
Copy link
Member

@RAMitchell I meant with trace. At least when bad_alloc happens we know where is the root, how much more memory do we need etc.

@trivialfis trivialfis merged commit 2fa8b35 into dmlc:master Sep 26, 2019
@trivialfis
Copy link
Member

@VibhuJawa Thanks.

@lock lock bot locked as resolved and limited conversation to collaborators Dec 26, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@VibhuJawa @trivialfis @RAMitchell